Method for adjusting an exposure of an endoscope

ABSTRACT

Methods, endoscopic systems, and non-transitory, machine-readable storage media for adjusting an exposure of an endoscopic system are described. In an embodiment, the methods include emitting light with a light source into a proximal end of an endoscope light pipe; generating received light signals with a photodetector based on light received through the endoscope light pipe by the photodetector; displaying, with a display module, images based on the received light signals having a current image brightness; monitoring the received light signals for changes to the current image brightness; and adjusting a light source illuminance level, and in some embodiments a photodetector gain and exposure time, based on the current image brightness to maintain a target image brightness of the display module. In an embodiment, a brightness adjustment step size is less than a just-noticeable difference in brightness of an eye and is directly proportional to the current image brightness.

CROSS-REFERENCE TO RELATED APPLICATIONS

This application is a continuation of U.S. application Ser. No.17/123,650, filed Dec. 16, 2020, which claims the benefit of U.S.Provisional Patent Application No. 62/951,716, filed Dec. 20, 2019,which are incorporated herein by reference in entirety.

TECHNICAL FIELD

This disclosure relates generally to endoscopes, and in particular butnot exclusively, relates to adjusting an exposure of an endoscope.

BACKGROUND INFORMATION

Endoscopes are currently used in a wide variety of medical procedures,such as endoscopy and laparoscopy, to both illuminate and visualizeinternal organs of a patient. An endoscope camera feed is typicallydisplayed on a monitor to guide a physician in conducting the procedure.

A camera technology referred to as automatic exposure or auto-exposureautomatically adjusts image brightness displayed on a monitor forviewing. However, conventional auto-exposure methods have been developedand fine-tuned for capturing scenes illuminated by external sources.

Conventional auto-exposure methods, such as those typically used inconsumer electronics (e.g. video cameras and phone cameras) are designedto capture natural scenes, such as those illuminated by an externallight source. Given a scene, these methods adapt to the externalillumination by optimizing camera controls (exposure time, aperture sizeand sensor gain) to capture a well-exposed video feed. Anatomical scenespresent a unique set of challenges, such as reflective instruments inproximity to a camera, for which such auto-exposure methods have notbeen optimized and, therefore, such conventional methods demonstratepoor performance.

Such methods typically do not work well for endoscope visualization fora number of reasons. Endoscope hardware is typically different thanconsumer camera hardware. Endoscopes typically do not have a camerashutter and this, therefore, restricts the camera controls that can beoptimized to only exposure time and sensor gain. Endoscopes typicallyproduce and control their own illumination. The auto-exposure methodsdesigned for consumer cameras assume some level of external illuminationand cannot control its level. As such, using an auto-exposure methodoptimized for consumer electronics does not benefit from the ability toactively change illumination level and preserve a good signal-to-noiseratio (SNR). Auto-exposure methods in consumer cameras are not designedto deal with challenges specific to medical procedures such as a shinyreflective instrument in the scene, instruments, and anatomy very close(e.g., <30 mm) to the camera, and operating in a closed cavity withsignificant surface reflectance. Consumer electronics capture videos fora different use case of post-visualization where captures can bepost-processed to fix any remaining brightness issues. In contrast,endoscope visualization is live typically at 60 frames per second rate.The auto-exposure method for endoscope visualization should to be ableto actively optimize screen brightness.

BRIEF DESCRIPTION OF THE DRAWINGS

Non-limiting and non-exhaustive embodiments of the invention aredescribed with reference to the following figures, wherein likereference numerals refer to like parts throughout the various viewsunless otherwise specified. Not all instances of an element arenecessarily labeled so as not to clutter the drawings where appropriate.The drawings are not necessarily to scale, emphasis instead being placedupon illustrating the principles being described.

FIG. 1 is a flow chart illustrating a process of adjusting an exposureof an endoscope, in accordance with an embodiment of the disclosure.

FIG. 2 is a flow chart illustrating another process of adjusting anexposure of an endoscope, in accordance with an embodiment of thedisclosure.

FIG. 3 is a flow chart illustrating another process of adjusting anexposure of an endoscope, in accordance with an embodiment of thedisclosure.

FIG. 4 a schematic illustration of an endoscopic system, in accordancewith an embodiment of the disclosure.

DETAILED DESCRIPTION

Embodiments of methods, systems, and non-transitory, machine-readablemedia for adjusting an exposure of an endoscope are described herein. Inthe following description numerous specific details are set forth toprovide a thorough understanding of the embodiments. One skilled in therelevant art will recognize, however, that the techniques describedherein can be practiced without one or more of the specific details, orwith other methods, components, materials, etc. In other instances,well-known structures, materials, or operations are not shown ordescribed in detail to avoid obscuring certain aspects.

Reference throughout this specification to “one embodiment” or “anembodiment” means that a particular feature, structure, orcharacteristic described in connection with the embodiment is includedin at least one embodiment of the present invention. Thus, theappearances of the phrases “in one embodiment” or “in an embodiment” invarious places throughout this specification are not necessarily allreferring to the same embodiment. Furthermore, the particular features,structures, or characteristics may be combined in any suitable manner inone or more embodiments.

The present disclosure provides methods, endoscopic systems, andnon-transitory, machine-readable media for use in endoscopevisualization pipelines. Typically, an endoscope not only serves as thecamera, or visualization element, but also as an illumination source fora surgical scene. As such, image quality produced by an endoscope can becontrolled not only by camera parameters but also by illuminationparameters.

Auto-exposure methods seek to maintain a user-specified target imagebrightness regardless of illumination level and object reflectance. Moreparticularly for surgical scenes, as the endoscope is moved to visualizedifferent anatomical regions and, potentially, surgical instruments inscene, the auto-exposure method should maintain a uniform level of imagebrightness in video feed, such as an image brightness based upon a userinput.

The methods of the present disclosure adjust an illuminance of a lightsource of an endoscopic system, an exposure time, and a sensor gain tomaintain an even image brightness, such as while also reducing asignal-to-noise ratio (SNR), as the endoscope is moved about a surgicalscene. In comparison, conventional auto-exposure methods, such as thosedeveloped for consumer video cameras, only optimize camera parameters(e.g. shutter speed, aperture size, and gain) as there is almost alwayssome level of external illumination. Such external illuminationtypically cannot be controlled or measured by a camera. As a result, indark scenes, image quality suffers from significant noise due to highexposure time and sensor gain.

In an embodiment, the methods of the present disclosure are suitable totransition between different control settings such that an imagebrightness change on the monitor is minimally noticeable frame-to-frame,such as to a human eye, and suitable to provide a smooth visualizationexperience. This is in contrast to conventional auto-exposure methodsthat result in scene flickering, oscillation, or flashing as a result ofchanging control settings. Such flickering, oscillation, or flashingadversely affects a user visualization experience because the uservisualization experience is used by surgeons, for example, to guidetheir procedures.

A method 100, in accordance with an embodiment of the presentdisclosure, will now be described with respect to FIG. 1 . FIG. 1 is aflow chart illustrating the method 100 of adjusting an exposure of anendoscope, in accordance with an embodiment of the disclosure.

As shown, method 100 can begin with process block 101 including emittinglight into an endoscope. In an embodiment, such emission includesemitting light with a light source into a proximal end of an endoscopelight pipe. At least a portion of such light is transmitted through theendoscope light pipe from the proximal end to a distal end of theendoscope light pipe, such as by total internal reflection. The portionof the light transmitted through the endoscope light pipe can be emittedfrom the distal end and onto a scene, such as a surgical scene inside ofa body. At least a portion of the light emitted onto the scene isreflected or scattered off of the scene and again received by the distalend of the endoscope light pipe.

The light source can be any light source suitable to emit light into theendoscope light pipe. The emitted light can include light having anywavelengths suitable to interrogate a scene, such as a surgical scene.In an embodiment, the light source is configured to emit light havingwavelengths in, for example, an ultraviolet range (e.g. in a range ofabout 10 nm to about 400 nm), a visible range (e.g. in a range of about400 nm to about 700 nm), or an infrared range (e.g. in a range of about700 nm to about 1 mm).

While light sources positioned at a proximal end of an endoscope lightpipe are described, it will be understood that the light source can bepositioned at other portions of an endoscopic system, such as at adistal end of an endoscope light pipe.

In an embodiment, process block 101 is followed by process block 103,which includes generating received light signals based on light receivedthrough the endoscope light pipe. In an embodiment, process block 103includes generating received light signals with a photodetector based onlight received through the endoscope light pipe by the photodetector,such as with a photodetector positioned at the proximal end of theendoscope light pipe. Such a photodetector may be responsive to orotherwise sensitive to light having one or more wavelengths emitted bythe light source, such as in generating the received light signals.While a photodetector positioned at the proximal end of the endoscopelight pipe is described, it will be understood that the photodetectorsof the present disclosure can be positioned in a number of positions inan endoscopic system, in an accordance with the embodiments describedherein, such as at a distal end of the endoscope light pipe.

In an embodiment, generating received light signals with a photodetectorincludes generating received light signals based on light received overa period of time. Such generation of received light signals can includegenerating a number of received light signals based on discrete periodsof time. In an embodiment, generation of received light signals includesgenerating received light signals based on sequential discrete periodsof time, such as received light signals suitable for display as a videofeed of a surgical scene. In this regard, in an embodiment, the receivedlight signals are generated at a frame rate, wherein received lightsignals are generated, for example, at regular intervals in sequence astime elapses.

In an embodiment, process block 103 is followed by process block 105,which includes displaying images based on the received light signals. Inan embodiment, displaying images based on the received light signalsincludes displaying, with a display module, images based on the receivedlight signals having a current image brightness. As discussed furtherherein, in an embodiment, the display module displays the images at acurrent image brightness based on a user-defined brightness, such as abrightness based on an input received from a user. In an embodiment, thedisplay module displays the images as a video feed of the scene.

In an embodiment, process block 105 is followed by process block 107,which includes monitoring the received light signals. As discussedfurther herein, the received light signals, images, and image brightnessbased thereon, can change as, for example, an orientation of the distalend of the endoscope light pipe and/or a scene captured by the distalend of the endoscope light pipe change.

In an embodiment, monitoring the received light signals includescapturing a raw image generated by the photodetector, such as a currentraw photodetector image. In an embodiment, monitoring the received lightsignals includes monitoring the received light signals for changes tothe received light signals, such as changes to current image brightness,such as current image brightness of images being displayed by thedisplay module.

In an embodiment, process block 107 is followed by process block 109,which includes calculating a current image brightness. In an embodiment,calculating a current image brightness includes calculating the currentimage brightness based on the received light signals. In an embodiment,process block 109 is optional. In an embodiment, a photodetectorresponse is proportional, such as linearly proportional, to at leastthree factors including a light source illumination level (L), aphotodetector integration or exposure time (IE), and a photodetectorgain (G).

Regarding the light source illumination level, an increase inintensity/power output of the light source increases a photodetectorresponse, such as without or without substantially adding noise to thereceived light signals. In an embodiment, a maximum illuminationintensity is limited by patient safety, such as the safety and integrityof a portion of a body exposed the light emitted from the distal end ofthe endoscope light pipe. As discussed further herein in greater detail,in an embodiment, a rate of illumination intensity change is limited byany latency associated with achieving stable power output as lightsource intensity or power output is changed.

Regarding the exposure time, an increase in the exposure, such as a timeinterval used by the photodetector to capture photons, increases aphotodetector response, such as without or without substantially addingnoise to the received light signals. In an embodiment, a maximumintegration time is limited by a frame rate of the photodetector ingenerating the received light signals.

Regarding gain, in an embodiment, the gain is a multiplicative factor tophotodetector response. In an embodiment, an increase in the gainincreases both signal and amplifies noise.

In an embodiment, current image brightness (B_(C)) can be according toEqn. (1):B _(C) ∝L _(C) ×IE _(C) ×G _(C)  Eqn. (1)where, L_(C), IE_(C) and G_(C) represent current illumination level,current integration time, and current gain, respectively. In anembodiment, B_(C) is calculated by performing a histogram analysis of araw sensor image, such as while ignoring a fraction of the tail end ofhistogram (representing highlights) to make auto-exposure resilient topresence of, e.g., reflective surgical instruments in the scene. In anembodiment, L_(C), IE_(C) and G_(C) are all endoscopic system settingsthat can be read directly from endoscope hardware, such as from thephotodetector and the light source.

In an embodiment, method 100 includes decision block 111, which includesdetermining a difference between a user-defined image brightness and acurrent image brightness, such as a current image brightness asdetermined in process block 109. Such a user-defined image brightnesscan be an image brightness, such as from an image displayed in a displaymodule, defined by an input from a user. As shown, if an absolute valueof a difference between the user-defined image brightness and thecurrent image brightness is over a predetermined threshold, the method100 may continue to further process blocks described herein. As alsoshown, if the absolute value of difference between the user-definedimage brightness and the current image brightness is under thepredetermined threshold, then, in an embodiment, the method 100 mayrevert to process block 103 in which the photodetector generatesreceived light signals.

In an embodiment, the predetermined threshold is suitably low tomaintain the image brightness to within acceptable ranges of theuser-defined image brightness as a distal end of the endoscope lightpipe moves about a surgical scene. In an embodiment, the predeterminedthreshold is preset on the endoscopic system. In an embodiment, thepredetermined threshold can be determined by a user, such as with a userinterface.

In an embodiment, decision block 111 is followed by process block 113,which includes determining a current light source illuminance level, acurrent photodetector exposure time, and a current photodetector gainlevel. In an embodiment, such a determination is performed by queryingone or more components of endoscopic system hardware, such as byquerying the photodetector, the light source, and/or a controlleroperatively coupled thereto. In an embodiment, process block 113 isoptional.

In an embodiment, process block 113 is followed by process block 115,which includes calculating a just-noticeable brightness difference. Ajust-noticeable brightness difference is a difference in imagebrightness that is just large enough to be detectable by an eye, such asby a human eye. As discussed further herein, it is a goal of the methodsof the present disclosure to avoid flickering, oscillation, flashing,and the like in adjusting an exposure of an endoscopic system. Suchflickering, oscillation, or flashing will degrade a user experience ofan endoscopic system. Accordingly, as also discussed further herein,endoscopic system parameters may be adjusted, such as iterativelyadjusted, in brightness adjustment step sizes less than thejust-noticeable brightness difference.

In an embodiment, the just-noticeable brightness difference iscalculated and/or defined according to the Weber-Fechner law, whichdefines the relationship between the just-noticeable difference inbrightness (dB_(C)) in terms of reference brightness (B) and a constant(K), such as according to Eqn. (2):dB=K×B  Eqn. (2)

This implies that if current brightness B_(C) is small, smaller stepsshould be taken towards achieving target image brightness. Similarly, ifthe current brightness B_(C) is high, larger steps can be taken towardsachieving target image brightness. In an embodiment, the constant, K,depends upon image brightness, wavelengths of light of the image, andthe like.

In an embodiment, process block 115 is followed by process block 117,which includes setting a target brightness. In an embodiment, targetbrightness can be defined according to Eqn (3):B _(T) ∝L _(T) ×IE _(T) G _(T)  Eqn. (3), wherein L_(T), IE_(T) and G_(T) represent target light sourceillumination level, photodetector integration or exposure time, andphotodetector gain, respectively. A goal of setting the targetbrightness is to find a combination of endoscopic system settings (e.g.L_(T), IE_(T), and G_(T)) such that target image brightness (B_(T)) canbe achieved with a low or minimized SNR.

In an embodiment, setting the target brightness includes setting thetarget image brightness based on the user-defined image brightness. Inan embodiment, process block 117 is optional. In an embodiment, thetarget brightness is defined according to Eqn. (4):B _(T) =B _(C)+sign(B _(U) −B _(C))×dB_(C)  Eqn. (4)wherein B_(U) is a user-defined image brightness.

In an embodiment, process block 117 is followed by process block 119,which includes calculating a target light source illuminance level, atarget exposure time, and a target gain level. In an embodiment, suchcalculations are based upon the current image brightness and the targetimage brightness, such as may be defined in process blocks 109 and 117,respectively. In an embodiment, process block 119 is optional.

In an embodiment, process blocks 109, 113, 115, 117, or 119 can befollowed by process block 121, which includes adjusting a light sourceilluminance level based on the current image brightness. In anembodiment, the light source illuminance level is one of a number ofendoscopic system settings further including one or more of an exposuretime of the photodetector and a gain of the photodetector, wherein themethod includes adjusting the endoscopic system settings based on thecurrent image brightness. Such adjustments are suitable to maintain thetarget image brightness of the display module, such as the receivedlight signals change.

In an embodiment, adjusting the light source illuminance level and/orlevels of other endoscopic system settings, such as a gain and anexposure time, include making brightness adjustment steps. Suchbrightness adjustment steps can be made iteratively. In an embodiment,the brightness adjustment step size is less than a just-noticeabledifference in brightness of an eye, such that as the endoscopic systemsettings are changed to maintain a target image brightness within arange not noticeable to an observer view a displayed image, and theimage brightness does not perceptibly change. In an embodiment, and asdiscussed further herein such as with respect to process block 115, sucha just-noticeable difference is based on current image brightness.Accordingly, in an embodiment, the brightness adjustment size step isdirectly proportional to the current image brightness.

In an embodiment, current image brightness is less than a target imagebrightness. Accordingly, attention is directed to FIG. 2 in which amethod 200 for adjusting an exposure of an endoscopic system, accordingto an embodiment of the disclosure, is illustrated.

As shown, method 200 can begin with process block 201, which includesemitting light with a light source into a proximal end of an endoscopelight pipe. In an embodiment, process block 201 is an example of processblock 101 discussed further herein with respect to method 100 and FIG. 1.

In an embodiment, process block 201 is followed by process block 203,which includes generating received light signals. In an embodiment,generating received light signals includes generating received lightsignals with a photodetector based on light received through theendoscope light pipe by the photodetector positioned at the proximal endof the endoscope light pipe. In an embodiment, process block 203 is anexample of process block 103 of method 100 discussed further herein withrespect to FIG. 1 .

In an embodiment, process 203 is followed by process block 205, whichincludes displaying images based upon received light signals. In anembodiment, displaying images based upon received light signals includesdisplaying, with a display module, images based on the received lightsignals having a current image brightness. In an embodiment, processblock 205 is an example of process block 105 discussed further hereinwith respect to FIG. 1 .

In an embodiment, process block 205 is followed by decision block 207,which includes determining whether the target light source illuminancelevel and the target exposure time are limited by predeterminedthresholds. In an embodiment, target image brightness, target lightsource illuminance level, target exposure time, target gain, and thelike are calculated according to the methods described further hereinwith respect to methods 100 and 300 discussed further herein withrespect to FIGS. 1 and 3 .

In an embodiment, the target light source illuminance level is limitedby patient safety. As discussed further herein, a threshold for a lightsource illuminance level can be limited where an illuminance level overthe threshold would damage patient tissue exposed to such an illuminancelevel.

If the target light source illuminance level and the target exposuretime are not limited by predetermined thresholds, process block 205 maybe followed by process block 209, which includes increasing the lightsource illuminance level and the exposure time while maintaining thegain at a constant level to achieve the target image brightness. In anembodiment, increasing the light source illuminance level and theexposure time includes simultaneously increasing the light sourceilluminance level and the exposure time. By maintaining a photodetectorgain at a constant level, such as at a minimum constant level or at acurrent gain level, SNR is increased or maximized. Such increasing ormaximizing of SNR is suitable for producing high-quality images fordisplay with the display module.

If the target light source illuminance level and/or the target exposuretime are limited by predetermined thresholds, process block 205 can befollowed by process block 211, which includes increasing the gain toachieve the target image brightness. In an embodiment, process block 211further includes maintaining the light source illuminance level and/orthe exposure time at or below the predetermined thresholds, such as toavoid damaging patient tissue.

Optimization of sensor integration time and illumination level can bedone in several different ways depending on hardware properties

If illumination level can be actively changed in a stable manner withoutdisrupting video rate, illumination level can be changed first wheremaximum illumination is limited by patient safety and minimumillumination is limited by image noise. If BT cannot be achieved bychanging illumination level, integration time is changed as a secondstep.

If illumination level has some latency associated with stable poweroutput, illumination level cannot be changed in every frame. For theframes where illumination level is settling down and stabilizing, abalancing integration time change is made to keep image brightness asuniform as possible.

As discussed further herein with respect to equations 10 and 11, whenL_(T) and IE_(T) calculated, such as with equation 9, are fractionalnumbers, they may be rounded off to the closest possible values.However, when L_(T) and IE_(T) are small values, such rounding off mayresult in significant overshoot or undershoot in image brightnessrelative to the target brightness B_(T). To prevent this, the systemchecks for a brightness overshoot or undershoot. As discussed withrespect to equation 10, a user-defined threshold can be used todetermine whether to round up or down in adjusting the illuminance andintegration time.

In an embodiment, a target image brightness is less than a current imagebrightness. In that regard, attention is directed to FIG. 3 in which amethod 300 of adjusting an exposure of an endoscopic system, inaccordance with an embodiment of the disclosure is illustrated.

As shown, method 300 begins with process block 301, which includesemitting light with a light source into a proximal end of an endoscopelight pipe. In an embodiment, process block 301 is an example of processblocks 101 and/or 201 discussed further herein with respect to FIGS. 1and 2 .

In an embodiment, process block 301 is followed by process block 303,which includes generating received light signals. In an embodiment,generating received light signals includes generating received lightsignals with a photodetector based on light received through theendoscope light pipe by the photodetector positioned at the proximal endof the endoscope light pipe. In an embodiment, process block 303 is anexample of process blocks 103 and/or 203 discussed further herein withrespect to FIGS. 1 and 2 , respectively.

In an embodiment, process block 303 is followed by process block 305,which includes displaying images based on the received light signals. Inan embodiment, displaying images based on the received light signalsincludes displaying, with a display module, images based on the receivedlight signals having a current image brightness. In an embodiment,process block 305 is an example of process blocks 105 and/or 205discussed further herein with respect to FIGS. 1 and 2 , respectively.

In an embodiment, process block 305 is followed by process block 307,which includes adjusting the gain, such as adjusting the gain to aminimum level. Using Eqns. (1) and (3),

$\begin{matrix}{\frac{B_{C}}{B_{T}} = \frac{L_{C} \times {IE}_{C} \times G_{C}}{L_{T} \times {IE}_{T} \times G_{T}}} & {{Eqn}.(5)}\end{matrix}$ $\begin{matrix}{{L_{T} \times {IE}_{T} \times G_{T}} = {L_{C} \times {IE}_{C} \times G_{C} \times \frac{B_{T}}{B_{C}}}} & {{Eqn}.(6)}\end{matrix}$

Accordingly, in an embodiment, a target gain, such as a minimum targetgain, is according to the Eqn. (7):

$\begin{matrix}{G_{T} = {{round}\left( \frac{RHS}{{IE}_{Max} \times L_{Max}} \right){off}{to}{the}{next}{higher}{possible}{gain}{value}}} & {{Eqn}.(7)}\end{matrix}$where RHS is defined by Eqn. (8) as

$\begin{matrix}{{RHS} = {L_{C} \times {IE}_{C} \times G_{C} \times \frac{B_{T}}{B_{C}}}} & {{Eqn}.(8)}\end{matrix}$

In an embodiment, adjusting the gain includes adjusting the gain to alowest possible value, such as according to Eqns. 5-8 and where thelight source illumination level and exposure time are not limited. In anembodiment, the light source illumination level is not limited by athreshold defined by a patient safety. In an embodiment, the exposuretime is not limited by a frame rate of the display module.

In an embodiment, process block 307 is followed by decision block 309,which includes determining whether there is a latency in adjusting alight source illuminance level, such as in adjusting a currentilluminance level to a target illuminance level. Whether a latencyexists can depend on a number of factors including one or more ofendoscopic device hardware including the light source, a frame rate ofobtaining images, and the like.

If the light source illuminance level can be actively changed, such asfrom a current light source illuminance level to a target light sourceilluminance level, in a stable manner without disrupting video rate,light source illuminance level should be changed first where maximumillumination is limited by patient safety and minimum illumination islimited by image noise. Accordingly, in an embodiment, decision block309 is followed by process block 311, which includes adjusting a lightsource illuminance level. In an embodiment, process block 311 includesadjusting the light source illuminance level such that adjusting thelight source illuminance level does not affect a frame rate andaccompanying exposure time of generating the received light signals withthe photodetector.

In an embodiment, process block 311 is followed by process block 313,which includes adjusting the exposure time after adjusting the lightsource illuminance level. In an embodiment, process block 313 isoptional, such as where adjusting the gain to a minimum level andadjusting the light source illuminance level is sufficient to obtain atarget image brightness and/or to achieve a suitable brightnessadjustment step size.

If changing the light source illuminance level has some latencyassociated with stable power output, a light source illuminance level isnot changed in every frame. Accordingly, in an embodiment, decisionblock 309 is followed by process block 315, which includes adjusting thelight source illuminance level on fewer than all images based on thereceived light signals.

In an embodiment, process block 315 is followed by process block 317,which includes adjusting the exposure time for images wherein the lightsource illuminance level is not adjusted. In this regard, for the frameswhere the light source illumination level is settling down andstabilizing, a balancing integration time change is made to keep imagebrightness as uniform as possible.

In an embodiment, once target gain (G_(T)) is calculated, targetintegration time (IE_(T)) and target illumination level (L_(T)) aresimultaneously optimized:

$\begin{matrix}{{L_{T} \times {IE}_{T}} = \frac{L_{C} \times {IE}_{C} \times G_{C} \times B_{T}}{G_{T} \times B_{C}}} & {{Eqn}.(9)}\end{matrix}$

where, G_(T) is the lowest sensor gain calculated in equation 7.

The simultaneous optimization of sensor integration time andillumination level can be done in several different ways depending onhardware properties

If illumination level can be actively changed in a stable manner withoutdisrupting video rate, illumination level can be changed first wheremaximum illumination is limited by patient safety and minimumillumination is limited by image noise. If B_(T) cannot be achieved bychanging illumination level, integration time is changed as a secondstep.

If illumination level has some latency associated with stable poweroutput, illumination level cannot be changed in every frame. For theframes where illumination level is settling down and stabilizing, abalancing integration time change is made to keep image brightness asuniform as possible.

When L_(T) and IE_(T) calculated from equation 9 are fractional numbers,they may be rounded off to the closest possible values. However, whenL_(T) and IE_(T) are small values, such rounding off may result insignificant overshoot or undershoot in image brightness relative to thetarget brightness B_(T). To prevent this, the system checks for abrightness overshoot or undershoot by calculating:

$\begin{matrix}{{❘{1 - \frac{{{round}\left( {IE}_{T} \right)} \times {{round}\left( L_{T} \right)}}{{IE}_{T} \times L_{T}}}❘} < \epsilon} & {{Eqn}.(10)}\end{matrix}$

where, ϵ is a user-defined threshold. If the condition is true, IE_(T)and L_(T) are rounded off to the closest possible values, which producesan image brightness close enough to target brightness B_(T). However, ifthis condition fails, IE_(T) and L_(T) are floored to the next lowervalues and sensor gain G_(T) is simultaneously increased to achieve thetarget brightness B_(T):

$\begin{matrix}{G_{T} = {{{round}\left( \frac{L_{C} \times {IE}_{C} \times G_{C} \times B_{T}}{{{floor}\left( L_{T} \right)} \times {{floor}\left( {IE}_{T} \right)} \times B_{C}} \right)}{off}{to}{the}{closest}{gain}{value}}} & {{Eqn}.(11)}\end{matrix}$

Accordingly, in an embodiment, the light source illuminance level andthe exposure time are rounded up when the light source illuminance leveland the exposure time are fractional numbers and a ratio of a product ofa rounded light source illuminance level and a rounded exposure time toa product of a fractional light source illuminance and a fractionalexposure time is below a user-defined threshold. Likewise, in anembodiment, the fractional light source illuminance and the fractionalexposure time are rounded down and the gain is increased to obtaintarget brightness when the ratio of the product of the rounded lightsource illuminance level and the rounded exposure time to the product ofthe fractional light source illuminance and the fractional exposure timeis above the user-defined threshold.

The order in which some or all of the process blocks appear in eachprocess should not be deemed limiting. Rather, one of ordinary skill inthe art having the benefit of the present disclosure will understandthat some of the process blocks may be executed in a variety of ordersnot illustrated, or even in parallel.

In another aspect, the present disclosure provides an endoscopic systemconfigured to adjust an exposure of the endoscopic system. In thatregard, attention is directed to FIG. 4 in which an endoscopic system400, in accordance with an embodiment of the disclosure, is illustrated.

As shown, the endoscopic system 400 includes an endoscope light pipe402, a light source 408, a display module 412, and a controller 416. Inthe illustrated embodiment, the endoscope light pipe 402 has a proximalend 404 and a distal end 406, wherein the light source 408 positioned toemit light into the proximal end 404. In an embodiment, the endoscopelight pipe 402 is flexible such that the distal end 406 can be movedabout a scene 414 to receive light 426 reflected and/or scattered off ofdifferent portions of the scene 414.

The photodetector 410 is positioned to receive light 426 from theendoscope light pipe 402, such as through the proximal end 404, such asthrough the optical switch 428. In an embodiment, the photodetector 410and other portions of the endoscopic system 400 do not include a shutteror other component configured to selectively occlude or limit lightreceived from the proximal end 404 from reaching the photodetector 410.In this regard, an exposure of the endoscopic system 400 is adjusted byadjusting one or more of, for example, the light source 408 illuminancelevel, the photodetector 410 gain, and the photodetector 410 exposuretime.

The display module 412 is shown operatively coupled to the photodetector410 and, in this regard, is configured to display an image of a scene414 based on light 426 received by the photodetector 410. The controller416 is shown to be operatively coupled to the light source 408, thedisplay module 412, and the photodetector 410. As such, the controller416 is shown in wired communication with the light source 408, displaymodule 412, and the photodetector 410, although wireless configurationsare also within the scope of the present disclosure.

The controller 416 includes logic that, when executed by the controller416, causes the endoscopic system 400 to perform operations. In anembodiment, such operations are suitable to perform one or more of themethods 100, 200, and 300 described further herein with respect to FIGS.1, 2, and 3 . In the illustrated embodiment, the controller 416 is shownto include illuminance control logic 420 for controlling a light source408 illuminance level, gain control logic 422 for controlling aphotodetector 410 gain level, and exposure control logic 424 forcontrolling a photodetector 410 gain.

In an embodiment, the controller 416 includes logic that, when executedby the controller 416, causes the endoscopic system 400 to performoperations including: emitting light 426 with the light source 408;generating received light signals with the photodetector 410 based onlight 426 received by the photodetector 410 from the proximal end 404 ofthe endoscope light pipe 402; displaying with the display module 412images based on the received light signals having a current imagebrightness; and adjusting a light source 408 illuminance level of thelight source 408 based on the current image brightness to maintain atarget image brightness.

In an embodiment, the light source 408 illuminance level is one of anumber of endoscopic system settings configured to be adjusted to adjustan exposure of the endoscopic system 400. In an embodiment, theendoscopic system settings further include one or more of an exposuretime of the photodetector 410 and a gain of the photodetector 410,wherein the operations include adjusting the endoscopic system settingsbased on the current image brightness. As described further herein withrespect to FIGS. 1-3 , such endoscopic system settings can be adjustedindependently or in tandem to adjust the exposure of the endoscopicsystem 400.

In an embodiment, a brightness adjustment step size is less than ajust-noticeable difference in brightness of an eye and is directlyproportional to the current image brightness. As discussed furtherherein with respect to process block 115 of FIG. 1 , by makingbrightness adjustment steps smaller than a just-noticeable difference inbrightness, a target image brightness is maintained as received lightsignals change.

In the illustrated embodiment, the endoscopic system 400 furtherincludes a user interface 418. In an embodiment, the user interface 418is configured to receive an input from a user, such as an input from auser for a user-defined image brightness. In this regard, in anembodiment, the endoscopic system 400 is configured to adjust anexposure time to maintain a target image brightness based on theuser-defined image brightness. In an embodiment, the controller 416further includes logic that when executed by the controller 416 causesthe endoscopic system 400 to perform operations including: calculatingthe current image brightness based on the received light signals;determining a current light source 408 illuminance level, a currentphotodetector 410 exposure time, and a current photodetector 410 gainlevel if a difference between the user-defined image brightness and thecurrent image brightness is above a predetermined image brightnessthreshold; calculating the just-noticeable brightness difference;setting a target image brightness based on the user-defined imagebrightness; and calculating a target light source 408 illuminance level,a target exposure time, and a target gain level based on the currentimage brightness and the target image brightness.

In an embodiment, the controller 416 includes logic for adjusting theendoscopic system settings for scenarios in which the current imagebrightness is less than the target image brightness. In an embodiment,adjusting the endoscopic system settings include simultaneouslyincreasing the light source 408 illuminance level and the exposure timewhile maintaining the gain at a constant level to achieve the targetimage brightness. In an embodiment, adjusting the endoscopic systemsettings includes increasing the gain to achieve the target imagebrightness, wherein the light source 408 illuminance level and theexposure time are limited by predetermined thresholds.

In an embodiment, the controller 416 includes logic for adjusting theendoscopic system settings for scenarios in which the current imagebrightness is greater than the target image brightness. In anembodiment, adjusting the endoscopic system settings includes adjustingthe gain to a minimum level; adjusting the light source 408 illuminancelevel, wherein adjusting the light source 408 illuminance level does notaffect a frame rate of generating the received light signals with aphotodetector 410; and adjusting the exposure time after adjusting thelight source 408 illuminance level. In an embodiment, adjusting theendoscopic system settings includes adjusting the gain to a minimumlevel; adjusting the light source 408 illuminance level on fewer thanall images based on the received light signals wherein adjusting thelight source 408 illuminance level has an adjustment latency; andadjusting the exposure time for images wherein the light source 408illuminance level is not adjusted.

The processes explained above are described in terms of computersoftware and hardware. The techniques described may constitutemachine-executable instructions embodied within a tangible ornon-transitory machine (e.g., computer) readable storage medium, thatwhen executed by a machine will cause the machine to perform theoperations described. Additionally, the processes may be embodied withinhardware, such as an application specific integrated circuit (“ASIC”) orotherwise.

In another aspect, the present disclosure provides non-transitory,machine-readable storage media having instructions stored thereon. Suchinstructions, when executed by a processing system, cause the processingsystem to perform operations, such as operations for performing one ormore of the methods of the present disclosure. In an embodiment, suchmethods include one or more of methods 100, 200, and 300 discussedfurther herein with respect to FIGS. 1-3 , respectively.

In an embodiment, the non-transitory, machine-readable storage mediumhas instructions stored thereon, which when executed by a processingsystem, cause the processing system to perform operations including:emitting light with a light source into a proximal end of an endoscopelight pipe; generating received light signals with a photodetector basedon light received through the endoscope light pipe by the photodetectorpositioned at the proximal end of the endoscope light pipe; displaying,with a display module, images based on the received light signals havinga current image brightness; monitoring the received light signals forchanges to the current image brightness; and adjusting a light sourceilluminance level based on the current image brightness to maintain atarget image brightness of the display module.

As discussed further herein with respect to, for example, FIG. 1 , thelight source illuminance level is one of a number of endoscopic systemsettings further including one or more of an exposure time of thephotodetector and a gain of the photodetector. In an embodiment, theoperations include adjusting the endoscopic system settings based on thecurrent image brightness.

In an embodiment, a brightness adjustment step size made in adjustingone or more of the endoscopic system settings is less than ajust-noticeable difference in brightness of an eye and is directlyproportional to the current image brightness.

In an embodiment, the non-transitory, machine-readable storage mediumincludes further instructions stored thereon, which when executed by aprocessing system, cause the processing system to perform operations toperform operations of method 100, such as discussed further herein withrespect to FIG. 1 , and, in particular, one or more process blocksselected from process blocks 113, 115, and 117. Accordingly, in anembodiment, the non-transitory, machine-readable storage medium includesinstructions stored thereon, which when executed by a processing system,cause the processing system to perform operations comprising:calculating a current image brightness based on the received lightsignals; determining a current light source illuminance level, a currentphotodetector exposure time, and a current photodetector gain level if adifference between a user-defined image brightness and the current imagebrightness is above a predetermined image brightness threshold;calculating the just-noticeable brightness difference; setting a targetimage brightness based on the user-defined image brightness; andcalculating a target light source illuminance level, a target exposuretime, and a target gain level based on the current image brightness andthe target image brightness.

A non-transitory, machine-readable storage medium includes any mechanismthat provides (i.e., stores) information in a non-transitory formaccessible by a machine (e.g., a computer, network device, personaldigital assistant, manufacturing tool, any device with a set of one ormore processors, etc.). For example, a machine-readable storage mediumincludes recordable/non-recordable media (e.g., read only memory (ROM),random access memory (RAM), magnetic disk storage media, optical storagemedia, flash memory devices, etc.).

The above description of illustrated embodiments of the invention,including what is described in the Abstract, is not intended to beexhaustive or to limit the invention to the precise forms disclosed.While specific embodiments of, and examples for, the invention aredescribed herein for illustrative purposes, various modifications arepossible within the scope of the invention, as those skilled in therelevant art will recognize.

These modifications can be made to the invention in light of the abovedetailed description. The terms used in the following claims should notbe construed to limit the invention to the specific embodimentsdisclosed in the specification. Rather, the scope of the invention is tobe determined entirely by the following claims, which are to beconstrued in accordance with established doctrines of claiminterpretation.

What is claimed is:
 1. A method of adjusting an exposure of anendoscopic system, the method comprising: emitting light with a lightsource into a proximal end of an endoscope light pipe; generatingreceived light signals with a photodetector based on light receivedthrough the endoscope light pipe by the photodetector; monitoring thereceived light signals for changes to a current image brightness; andadjusting a light source illuminance level and an exposure time of thephotodetector based on the current image brightness to maintain a targetimage brightness wherein the light source illuminance level and theexposure time are rounded up when the light source illuminance level andthe exposure time are fractional numbers and a ratio of a product of arounded light source illuminance level and a rounded exposure time to aproduct of a fractional light source illuminance and a fractionalexposure time is below a user-defined threshold, and wherein thefractional light source illuminance and the fractional exposure time arerounded down and a gain of the photodetector is increased to obtain atarget brightness when the ratio of the product of the rounded lightsource illuminance level and the rounded exposure time to the product ofthe fractional light source illuminance and the fractional exposure timeis above the user-defined threshold.
 2. The method of claim 1, whereinthe light source illuminance level is one of a number of endoscopicsystem settings further including the gain of the photodetector, whereinthe method includes adjusting the endoscopic system settings based onthe current image brightness.
 3. The method of claim 2, wherein thecurrent image brightness is less than the target image brightness, andwherein adjusting the endoscopic system settings comprises:simultaneously increasing the light source illuminance level and theexposure time while maintaining the gain at a constant level to achievethe target image brightness.
 4. The method of claim 2, wherein thecurrent image brightness is less than the target image brightness, andwherein adjusting the endoscopic system settings comprises: increasingthe gain to achieve the target image brightness wherein the light sourceilluminance level and the exposure time are limited by predeterminedthresholds.
 5. The method of claim 2, wherein the current imagebrightness is greater than the target image brightness, whereinadjusting the endoscopic system settings comprises: adjusting the gainto a minimum level; adjusting the light source illuminance level,wherein adjusting the light source illuminance level does not affect aframe rate of generating the received light signals with thephotodetector; and adjusting the exposure time after adjusting the lightsource illuminance level.
 6. The method of claim 2, wherein the currentimage brightness is greater than the target image brightness, whereinadjusting the endoscopic system settings comprises: adjusting the gainto a minimum level; adjusting the light source illuminance level onfewer than all images based on the received light signals, whereinadjusting the light source illuminance level has an adjustment latency;and adjusting the exposure time for images wherein the light sourceilluminance level is not adjusted.
 7. The method of claim 1, wherein abrightness adjustment step size is less than a just-noticeabledifference in brightness of an eye and is directly proportional to thecurrent image brightness.
 8. The method of claim 7, further comprising:calculating the current image brightness based on the received lightsignals; determining a current light source illuminance level, a currentphotodetector exposure time, and a current photodetector gain level if adifference between a user- defined image brightness and the currentimage brightness is above a predetermined image brightness threshold;calculating the just-noticeable brightness difference; setting a targetimage brightness based on the user-defined image brightness; andcalculating a target light source illuminance level, a target exposuretime, and a target gain level based on the current image brightness andthe target image brightness.
 9. An endoscopic system comprising: anendoscope light pipe having a proximal end and a distal end; a lightsource positioned to emit light into the proximal end; a photodetectorpositioned to receive light from the endoscopic light pipe; and acontroller operatively coupled to the light source and thephotodetector, the controller including logic that, when executed by thecontroller, causes the endoscopic system to perform operationsincluding: emitting light with the light source; generating receivedlight signals with the photodetector based on light received by thephotodetector from the proximal end of the endoscope light pipe; andadjusting a light source illuminance level of the light source and anexposure time of the photodetector based on a current image brightnessto maintain a target image brightness, wherein the light sourceilluminance level and the exposure time are rounded up when the lightsource illuminance level and the exposure time are fractional numbersand a ratio of a product of a rounded light source illuminance level anda rounded exposure time to a product of a fractional light sourceilluminance and a fractional exposure time is below a user-definedthreshold, and wherein the fractional light source illuminance and thefractional exposure time are rounded down and a gain of thephotodetector is increased to obtain a target brightness when the ratioof the product of the rounded light source illuminance level and therounded exposure time to the product of the fractional light sourceilluminance and the fractional exposure time is above the user-definedthreshold.
 10. The endoscopic system of claim 9, wherein the lightsource illuminance level is one of a number of endoscopic systemsettings further including the gain of the photodetector, wherein themethod includes adjusting the endoscopic system settings based on thecurrent image brightness.
 11. The endoscopic system of claim 10, whereinthe current image brightness is less than the target image brightness,and wherein adjusting the endoscopic system settings comprises:simultaneously increasing the light source illuminance level and theexposure time while maintaining the gain at a constant level to achievethe target image brightness.
 12. The endoscopic system of claim 10,wherein the current image brightness is less than the target imagebrightness, and wherein adjusting the endoscopic system settingscomprises: increasing the gain to achieve the target image brightness,wherein the light source illuminance level and the exposure time arelimited by predetermined thresholds.
 13. The endoscopic system of claim10, wherein the current image brightness is greater than the targetimage brightness, wherein adjusting the endoscopic system settingscomprises: adjusting the gain to a minimum level; adjusting the lightsource illuminance level, wherein adjusting the light source illuminancelevel does not affect a frame rate of generating the received lightsignals with a photodetector; and adjusting the exposure time afteradjusting the light source illuminance level.
 14. The endoscopic systemof claim 10, wherein the current image brightness is greater than thetarget image brightness, wherein adjusting the endoscopic systemsettings comprises: adjusting the gain to a minimum level; adjusting thelight source illuminance level on fewer than all images based on thereceived light signals wherein adjusting the light source illuminancelevel has an adjustment latency; and adjusting the exposure time forimages wherein the light source illuminance level is not adjusted. 15.The endoscopic system of claim 9, wherein a brightness adjustment stepsize is less than a just-noticeable difference in brightness of an eyeand is directly proportional to the current image brightness.
 16. Theendoscopic system of claim 15, wherein the controller further includeslogic that when executed by the controller causes the endoscopic systemto perform operations including: calculating the current imagebrightness based on the received light signals; determining a currentlight source illuminance level, a current photodetector exposure time,and a current photodetector gain level if a difference between a user-defined image brightness and the current image brightness is above apredetermined image brightness threshold; calculating thejust-noticeable brightness difference; setting a target image brightnessbased on the user-defined image brightness; and calculating a targetlight source illuminance level, a target exposure time, and a targetgain level based on the current image brightness and the target imagebrightness.
 17. A non-transitory, machine-readable storage medium havinginstructions stored thereon, which when executed by a processing system,cause the processing system to perform operations comprising: emittinglight with a light source into a proximal end of an endoscope lightpipe; generating received light signals with a photodetector based onlight received through the endoscope light pipe by the photodetector;monitoring the received light signals for changes to a current imagebrightness; and adjusting a light source illuminance level and anexposure time of the photodetector based on the current image brightnessto maintain a target image brightness wherein the light sourceilluminance level and the exposure time are rounded up when the lightsource illuminance level and the exposure time are fractional numbersand a ratio of a product of a rounded light source illuminance level anda rounded exposure time to a product of a fractional light sourceilluminance and a fractional exposure time is below a user-definedthreshold, and wherein the fractional light source illuminance and thefractional exposure time are rounded down and a gain of thephotodetector is increased to obtain a target brightness when the ratioof the product of the rounded light source illuminance level and therounded exposure time to the product of the fractional light sourceilluminance and the fractional exposure time is above the user-definedthreshold.
 18. The non-transitory, machine-readable storage medium ofclaim 17, wherein the light source illuminance level is one of a numberof endoscopic system settings further including the gain of thephotodetector, and wherein the operations include adjusting theendoscopic system settings based on the current image brightness. 19.The non-transitory, machine-readable storage medium of claim 17, whereina brightness adjustment step size is less than a just-noticeabledifference in brightness of an eye and is directly proportional to thecurrent image brightness.
 20. The non-transitory, machine-readablestorage medium of claim 19, further instructions stored thereon, whichwhen executed by a processing system, cause the processing system toperform operations comprising: calculating a current image brightnessbased on the received light signals; determining a current light sourceilluminance level, a current photodetector exposure time, and a currentphotodetector gain level if a difference between a user- defined imagebrightness and the current image brightness is above a predeterminedimage brightness threshold; calculating the just-noticeable brightnessdifference; setting a target image brightness based on the user-definedimage brightness; and calculating a target light source illuminancelevel, a target exposure time, and a target gain level based on thecurrent image brightness and the target image brightness.